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1.  Intruductiun 

I 

The  result  of  our  research  is  a  suite  ot  practical,  implementable  algorithms  that  a  software  dev  eloper  can 
use  to  build  a  high-throughput,  survivable  Code-Division  Multiple  Access  (CDMAi  mobile  packet  radio 
network  with  existing  spread-spectrum  packet-radio  hurdwiu'e.  '^he  Code  Division  channel  increases  net¬ 
work  throughput  by  allowing  simultaneous  transmissions  to  dil'terent  destinations  on  the  same  frequency 
band.  We  have  conducted  extensive  simulations  to  verify  the  performance  of  these  protocols  and  we  have 
determmed  they  produce  a  high-throughput,  survivable  network.  These  protocols  are  an  evolutionary  step 
in  packet-radio  network  architecture  that  starts  with  earlier  pioneering  work,  most  notably  the  work  of 
Jubin  et  al.  that  is  the  basis  of  SCRAP We  have  adopted  many  SCRAP  algorithms  to  our  CD.VLA 
network  architecture,  most  notably  its  routing  algorithms.  -  ^  r 

This  work  has  been  influenced  to  some  extent  by  a  particular  application.  Our  CDVLA  algorithms 
are  mtended  for  the  Low-cost  Packet-Radio  (LPR)  hardware  [4]  that  has  several  advanced  feamres.  most 
notably  its  power  level  and  FEC  rate  can  be  adjusted  on  a  packet-by-packet  basis  and  it  uses  forward 
error  correction  (FEC)  codes  to  compensate  for  bit  errors.  (The  salient  features  of  the  LPR  hardware 
are  summarized  in  Appendix  C.)  Our  algorithms  can  be  implemented  on  other  spread-spectrum  packet 
radio  devices  but  the  network  performance  will  be  affected  by  the  capabilities  of  the  particular  hardware. 
Nevertheless,  the  algorithms  in  this  paper  will  produce  a  high-quality  packet-radio  network  with  any 
spread-spectrum  packet  radio  device  that  can  u,se  multiple  spreading  codes. 

Conventional  spread-spectrum  packet-radio  networks  such  as  the  packet  overlay  for  SfNCGARS  [5] 
and  the  PRnet  with  the  SURAN  Protocol  (SURAP  (21).  do  not  use  code  division;  they  broadcast  their 
transmissions  on  a  single  common  radio  channel.  This  broadcast  channel  is  an  efficient  way  for  packet 
radios  to  exchange  routing  information  and  it  makes  network  entry  simple,  but  unfortunately  it  has  the 
undesirable  side-effect  of  causing  packet  radios  to  receive  packets  that  are  not  addressed  to  them.  Because 
a  packet  radio  cannot  determine  the  identity  of  a  received  packet  until  it  is  processed,  these  ‘’unwanted” 
receptions  increase  the  packet  radios'  processing  loads.  And.  because  conventional  packet  radio  hardware 
cannot  receive  more  than  one  packet  at  a  time,  when  transmissions  overlap  the  first  transmission  can  be 
received  but  all  others  are  blocked  until  this  reception  concludes.  Thus  the  broadcast  channel  increases 
the  fraction  of  time  that  the  packet  radio  nodes  are  in  this  blocking  state  and  thereby  increases  the  packet 
blocking  rate  and  consumes  channel  bandwidth  with  retransmissions.  A  CDMA  channel  on  the  other 
hand  has  much  lower  blocking  rates  and  thus  higher  throughput  because  the  nodes  can  be  selective  about 
which  packets  they  receive. 

Although  code  division  and  code  division  multiple  access  (CDMA)  are  familiar  concepts  to  com¬ 
munications  researchers,  to  our  knowledge  a  thorough  CDMA  network  design  has  never  been  attempted 
before.  In  thio  paper  we  summarize  the  pert'ormance  constraints  inherent  in  both  broadcast  and  CDMA 
multi-hop  packet-radio  net  works  and  present  the  results  of  a  snidy  that  compares  the  throughput  of  two 
protocol  suitc.s,  the  Broadcast  SURAP  protocol  suite  and  our  new  CDMA  protocol  suite.  Our  results 
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show  that  in  general  CDMA  can  signilicantly  improve  the  throughput  of  a  packet-radio  network  cuid  in 
particular  our  CDM.A  protocols  realize  this  performance  improvement,  provide  many  of  the  advantages  of 
a  broadcast  channel  and  provide  other  benehts  as  well,  such  as  effective  congestion  cont.ol  and  max/mm 
fair  end-to-end  flow  control. 

The  work  in  this  paper  presents  results  from  a  research  project  that  investigated  link-level  and  end- 
to-end  tlow  control  algorithms  for  CDMA  packet  radio  networks.  Our  main  objective  is  to  document 
new  link-level  algonthms.  but  the  pert'ormance  properties  of  a  packet-radio  network  are  determined  by  all 
its  constituent  algorithms;  the  link-layer  algorithms  are  only  one  component  of  a  much  larger  algorithm 
suite.  The  need  to  establish  a  conte.xt  for  our  performiuice  results  has  compelled  us  to  broaden  the  focus 
of  our  paper  to  include  .some  non-link-level  aspects  of  CDNLA  packet-radio  network  .u'chitecture. 

.A  noteworthy  o.xample  is  the  end-to-end  tlow-control  algonthm.  Our  CDMA  network  lU'chitecmre 
u.ses  a  new  packet-radio  congestion-control  algorithmfhj  (Appendix  .A)  that  dynamically  adjusts  end-to- 
end  rtow  rates.  The  throughput  and  sun.  ivability  improvements  we  report  are  due  mostly  to  the  link-layer 
algonthms.  whereas  the  fairness  improvements  are  due  mostly  to  the  end-to-end  congestion-control.  We 
will  indicate  the  source  of  unprovement  whenever  possible.  In  addition  we  are  not  the  architects  of  the 
link-gain-adjustment  algorithms  (Appendix  Bi  that  determine  power  levels.  FEC  rates  and  bit  rates.  We 
mention  these  algonthms  brietly  for  the  sake  of  completeness  and  refer  the  reader  to  a  more  thorough 
paper  [’]  on  this  topic. 

Our  paper  is  organized  in  the  following  way.  In  Chapter  (2)  we  explain  the  fundamental  performance 
factors  and  the  basic  link-layer  design  issues  for  packet  radio  networks.  Then  we  give  an  overview 
of  our  CDMA  network  architecture  followed  by  a  summary  of  SCRAP.  We  conclude  this  chapter  with 
a  comparison  of  the  two  architectures  that  introduces  the  performance  issues  we  investigated  with  our 
simulatton.  Chapter  (3i  gives  the  results  of  our  performance  study  and  Chapter  (4)  summarizes  these 
results  .md  makes  recommendations  for  fumre  research. 
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2.  Link-Layer  Protocols  for  Packet-Radio  Networks 

A  link-layer  protocol  provides  reliable  internode  data-packet  transport  by; 

1.  retransmitting  errored  or  blocked  data  packet  transmissions. 

2.  adjusting  link  gains  to  control  bit-error  rates. 

3.  adjusting  link  flow  rates  to; 

lai  stop  the  spread  of  congestion,  and 
ibi  maintam  channel  stability. 

These  vital  functions  are  achieved  by  distributed  algonthms  that  use  control  information  obtained  from 
a  dialogue  that  consists  of  transmitting  and  receiving  data  and  acknowledgement  packets.  Designing 
a  link-layer  protocol  for  packet-radio  networks  is  a  formidable  challenge  because  the  links  are  often 
unreliable,  their  bit-error  rates  are  highly  nonstationarv'.  and  packet  transmissions  are  frequently  blocked. 

In  the  next  section  we  explain  the  performance  factors  that  impact  link-layer  design  for  packet  radio 
networks.  .Later,  we  outline  our  CDMA  ;md  the  SURAP  link-layer  protiKols  and  conclude  this  section 
with  a  comparison  of  the  two  designs. 

2.1  Packet-Raclio-Network  Performance  Factors 

An  important  concept  that  differentiates  packet-radio  networks  from  other  multiple-access  networks,  e.g. 
Ethernet  LANs,  is  spatial  reuse.  Each  packet  radio  belongs  to  a  neighborhood  that  contams  all  the  packet 
radios  in  its  radio  range.  When  a  packet-radio  network  spans  a  large  geographic  region  and  there  are 
packet  radios  that  are  out  of  range  of  each  other  because  of  signal  attenuation,  the  region  will  be  covered 
by  overlapping  neighborhoods.  Then  the  common  radio  channel  has  spatial  reuse  in  the  sense  that  a 
packet  radio  can  communicate  with  any  radios  in  its  own  neighborhood  without  affecting  packet  radios 
M  other  isolated  neighborhoods. 

If  a  packet  radio  network  did  not  employ  spatial  reuse  all  the  radios  would  be  in  the  same  neighborhood 
and  the  network  would  be  topologically  equivalent  to  a  LAN.  In  most  cases  maintaining  this  topology 
would  require  transmissions  at  very  high  power  levels.  But  a  packet  radio  cannot  receive  packets  when 
overlappmg  transmissions  have  too  much  power.  This  c.ui  lower  the  signal-io-noise  level  at  a  packet 
radio  so  much  that  it  loses  synchromzation  or  the  received  packet  is  so  corrupted  with  bit  errors  that  FEC 
algorithms  are  ineffective.  This  performance  unpairment  is  called  mutual  interference. 

The  uncoupling  created  by  spatial  reuse  can  increa.se  a  network's  overall  throughput  by  increasing  the 
number  of  simultaneous  transmissions  and  decreasing  mutual  interference.  But.  as  we  will  soon  show. 
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spatial  reuse  creates  undesirable  interactions  between  overlapping  neighborhoods  that  decrease  network 
throughput. 

Consider  Figure  2.1  where  an  arrow  between  two  nodes  indicates  they  are  in  mutual  radio  range.  The 
indicated  connectivity  implies  packet  radio  .-i'  can  hear  the  transmissions  of  radios  X.  Z,  and  3  but 
neither  J  nor  T.  The  tirst  four  are  in  .-i  s  neighborhood;  the  latter  two  are  not.  We  say  that  radios  J  ;uid 
r  are  hidden  from  radio  A  and  radio  A  is  hidden  from  radios  J  and  Z.  In  general  radios  that  are  not  in 
range  are  hidden  from  each  other.  Thus  whenever  packet-radio  node  Z  is  transmitting  ito  node  C).  node 


Figure  2.T.  Packet-Radio-Network  Connectivity 

3  will  not  receive  these  transmissions  and  instead  it  can  receive  transmissions  from  node  A —  concurrent 
successful  trtuismissi.ons  increase  network  throughput.  On  the  other  hand,  a  packet  radio  cannot  receive 
two  transmissions  at  once  neither  can  it  receive  while  it  is  transmitting,  but  such  concurrent  transmissions 
can  occur  whenever  there  are  hidden  packet  radio  nodes.  For  example,  whenever  packet  radio  Z  is 
receiving  from  packet  radio  3.  packet  radio  3  cannot  detect  this  and  it  may  transmit  (unsuccessfully)  to 
packet  radio  3.  We  say  these  transmissions  are  blocked.  Blocking  wastes  bandwidth  and  the  ineffective 
radio  transmissions  increase  mutual  interference. 

CD.MA  and  broadcast  networks  are  both  affected  by  blocking  and  mutual  interference  to  varying 
degrees  but  the  precise  way  these  performance  factors  impact  network  performance  is  difficult  to  quantify. 

In  a  previous  paper  Zavgren  and  Lauer  [81  simulated  packet-radio  node  placement  and  routmg  to 
determine  the  mapping  between  end-to-end  hows  and  link-layer  flows  for  randomly-generated  network 
topologies.  Then  they  applied  an  analytic  model  to  particular  topologies  to  determine  these  networks' 
maximum  aggregate  end-to-end  throughputs  (capacity)  and  the  mutual  interference  generated  noise-power 
level  at  individual  nodes.  Their  work  showed  that  a  CDM.\  network  can  have  tive  times  the  capacity 
ot  an  identical  network  with  a  broadcast  channel;  furthermore.  CDMA  networks  are  less  susceptible 
to  jamming  than  broadcast  networks  because  their  lower  blocking  levels  requue  fewer  retransmission.'^, 
decreasing  mutual  interference.  The  performance  gam  is  topology  dependent  and  closed-form  analytic 
results  are  only  obtainable  when  networks  w  ith  significant  (i.e.,  atypical  and  unrealistic)  symmetry  are 
analyved.  Nevertheless,  we  can  conclude  that  the  denser  the  packet  radio  placement,  the  greater  the 
relative  throughput  improvement.  The  simulation  results  in  Section  (3)  are  consistent  with  this  earlier 
study,  however  the  magnitude  of  the  performance  gains  are  less  than  predicted  in  some  cases  because 
the  earlier  study  did  not  include  the  effects  of  end-to-end  flow  control.  Aggregate  end-to-end  uuoughput 
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can  be  decreased  when  max/min  tair  end-to-end  rtow  control  is  provided. 

The  tollowing  two  sections  outline  the  CDMA  and  the  SCRAP  link  layer  protocols  and  e.xplain 
two  important  components  tor  each;  link-level  acknowledgements.  ;md  link-level  congestion  avoidance. 
Link-level  acknowledgements  are  used  to  verily  the  reception  of  individual  packets;  unackne  wledged 
transmissions  are  repeated.  The  design  of  the  mechanisms  for  transmitting  and  receiving  link-level  ac¬ 
knowledgements  is  very  tightly  coupled  to  the  physical  layer.  CDMA  networks  requae  the  transmission 
of  a  separate  acknowledgement  packet  —  called  an  active  acknow  ledgement  —  whereas  broadcast  net¬ 
works  use  a  more  subtle  mechanism  —  called  a  pa.ssive  acknowledgement  that's  based  on  overheard  data 
packet  transmissions.  Link-level  congestion-avoidance  algorithms  are  designed  to  prevent  the  spread  of 
congestion  by  temporarily  restricting  the  utilization  of  critical  resources,  e.g..  packet  buffers  and  the  radio 
channel. 

2.2  (.'DM.A  Network.s 

2.2.1  Link-LeNel  Xcknowledgement.s 

Our  CDMA  link-level  protocol  is  based  on  the  assumption  that  every  packet  radio  is  assigned  a  unique 
spreading  code  that  it  uses  for  receiving  data  and  acknowledgement  packets  —  transmissions  on  other 
spreading  codes  cannot  be  dete'-ted.  This  section  gives  the  protocol  that  controls  the  e.xchange  of  data 
and  acknowledgement  packets  between  packet  radios.  Its  based  on  the  Preemptable  Moment  of  Silence 
Algorithm  [91.  which  uses  two  basic  principles; 

•  It  forces  a  data  packet  transmitter  to  defer  all  additional  data-packet  mansmissions  until  it  has 
monitored  the  channel  for  a  short  time  interval  called  a  VIoment  of  Silence. 

e  It  forces  a  data  packet  receiver  to  e.xpedite  its  formulation  of  an  acknowledgement  packet  and  to 
transmit  this  packet  at  a  pnonty  level  that  exceeds  that  of  all  data  packets. 

As  a  result,  the  transmitter  of  a  data  packet  gives  higher  priority  to  receiving  an  acknowledgement 
than  to  sending  another  data  packet  and  the  receiver  of  a  data  packet  is  more  likely  to  transmit  its 
acknowledgement  packet  when  the  data-packet  transmitter  is  listening.  This  algorithm  is  called  the 
Preemptable  Moment  of  Silence  algonthm  because  either  node  can  have  its  Moment  of  Silence  preempted 
by  a  data  packet  reception. 

Figure  2.2  illustmtes  a  successful  data-packet  transmission.  The  panuneter  r„, establishes  the  max¬ 
imum  duration  of  a  Moment  of  Silence.  If  an  acknowledgement  is  not  received  before  time  units 
have  elapsed,  the  Moment  of  Silence  is  terminated.  Otherwise,  it  ends  w  ith  the  reception  of  an  acknowl¬ 
edgement  packet.  The  value  of  is  a  design  parameter  that  depends  on  the  distribution  of  processing 
time.  Large  values  of  decrease  both  (he  acknowledgement-packet  blocking  rate  and  the  maximum 
data-packet  transmission  rate  —  small  values  increase  both.  Because  the  data-packet  proccssmg  times  in 
the  LPR  are  es.sentially  deterministic,  the  optimum  setting  is  slightly  larger  than  the  sum  of  the  min  mum 
data-packet  processing  time  and  the  minimum  acknowledgement-packet  transmission  time. 

The  Moment  of  Silence  does  not  prevent  acknowledgement-packet  blocking  but  it  does  make  the 
probability  of  this  event  extremely  small.  Becau.se  acknowledgement  packets  and  data  packets  lue  received 
on  the  same  spreading  codes,  occasionally  a  mxJe's  Moment  of  Silence  will  be  preempted  by  a  data-packet 
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Figure  2.2:  An  Acknowledged  Data  Packet  Transmission 


reception  that  blocks  an  acknowledgement  reception.  But  this  happens  infrequently  and  when  it  does  the 
data  packet  transmission  is  repeated. 

The  .Moment  of  Silence  algonthm  produce.s  an  efficient  link-layer.  It  increases  throughput  by  mak¬ 
ing  the  blockmg  rate  for  acknowledgement  transmissions  extremely  low.  A  useful  by-product  of  this 
algorithm  is  a  low-Jelay  high-reliability  "conduit"  for  acknowledgements.  We  have  utilized  this  property 
by  generalizing  the  concept  of  an  acknowledgement  packet  into  a  control  pucker.  These  packets  not 
only  acknowledge  the  reception  of  data  packets  to  provide  a  reliable  link,  they  also  ore  used  for  tiow 
control,  link-gain  adjustment,  and  the  distribution  of  end-to-end  congestion  control  inf'ormation[6].  We 
define  three  types  of  control  packets:  acknowledgment  packets,  flow-control  packets,  and  gam-adjustment 
packets.  Exactly  one  of  these  packets  is  transmitted  after  every  data-packet  reception.  When  a  packet 
radio  receives  a  data  packet  and: 

•  buffers  it  (i.e.,  all  bit  errors  are  correctable  and  a  buffer  is  available),  it  transmits  an  acknowledgment 
packet 

•  discards  it  because  it  has  no  buffers  available,  it  transmits  a  Jfow  control  packet 

•  discards  it  because  it  carmot  he  error-corrected,  it  transmits  a  gain-adiustmem  packet.  This  is 
the  case  if  a  packet  header  is  received  without  errors  but  the  body  is  not.  iWe  advocate  the 
encapsulation  of  the  ID  of  the  sender  w  ith  a  strong  error-correcting  code  so  gain  adjustment  packets 
can  be  transmitted  when  needed,  but  ihi.s  feature  i.s  not  mcluded  m  our  simulation.) 

Thus,  the  type  of  control  packet  transmitted  is  determined  by  what  a  packet  radio  does  with  each  received 
data  packet.  A  control -packet  summary  is  provided  in  Table  (2.1). 
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Link-Level  Control  Packets 

Type 

Purpose 

.Acknow  ledgemcnt 

.Acknowledges  the  reception  ind  buffer¬ 
ing  of  the  neighbor  s  data  packet. 

Flow  Control 

Informs  the  neighbor  that  its  data  packet 
wtis  received  but  discarded  due  to  msuf- 

ticient  buffers. 

(Tun  Adjustment 

Informs  the  neighbor  its  data  packet  vvas 
leceived  but  could  not  be  error  corrected 

and  was  discarded. 

Table  2.1:  Link-Level  Control  Packets 


2.2.2  Congestion  \\oi(lante  (Ne«  (  DM.\  Algorithms) 

We  employ  two  cooperating  congcstion-a\»>idance  mechanisms.  One  is  an  enJ-ro-cnd  rtovc -control  al¬ 
gorithm  that  establi.she.s  long-term  —  relative  to  the  length  ot'  a  tew  packet  transmissions  —  max/min 
t'air  end-to-end  tlow  rates,  that  is  summ,irt/ed  in  Appendi.x  .A.  In  this  section  we  explain  the  liiiic-lcvcl 
congestion-avoidance  algorithms  that  serve  three  important  purposes:  facilitate  the  flow  of  data  packets, 
protect  critical  nodal  resources  from  overload  during  short-n'vd  traffic  hursts,  and  provide  a  'safety  net " 
to  prevent  the  spread  of  congestion  when  the  end-to-end  tlow-control  algonthm  is  adapting  to  a  chiuige 
in  traffic  patterns. 

The  relationship  between  these  two  congestion-avoidance  mechanisms  has  two  aspects.  First,  the 
end-to-end  mechanism  takes  long-term  actions  that  anticipate  the  on.set  of  congestion  to  prevent  it  from 
occunng,  but  the  link-level  mechanism  takes  short-term  actions  that  react  to  congestion  and  quench  it. 
Second,  the  end-to-end  mechanism  regulates  the  flow  of  packets,  whereas  the  link-level  mechanism  in 
Its  normal  mode  of  operation,  expedites  the  flow  of  packets. 

The  CDMA  network  uses  its  link-level  congestion-avoidance  mechanisms  only  when  network  re¬ 
sources  are  exhausted.  Otherwise  —  that  is,  if  link-level  mechanisms  are  used  to  establish  Imk-level  flow 
rates  —  the  requirements  of  link-layer  flows  would  often  conflict  with  those  of  end-to-end  flows,  resulting 
m  unfair  end-to-end  flow  allocations.  ALso.  this  would  bias  the  measurements  used  by  the  end-to-end 
algonthm  and  result  in  even  more  unfairness.  The  CDMA  link-level  congestion-avoidance  mechanisms 
do  not  alleviate  congestion:  instead,  they  temporarily  restrict  access  to  resources  -.o  that  congestion  does 
not  spread  to  other  portums  of  the  network. 

Because  the  bandwidth-delay  priduct  for  the  LPRnet  is  small,  our  algoriih/iis  use  a  link-level  window 
size  of  one.  that  is,  a  node  is  inhibited  from  transmitting  a  data  packet  to  a  neighbor  until  it  receives 
an  acknowledgement  for  its  most-recent  data-packet  transmission  or  this  packet  is  discarded.  Our  three 
link-level  mechanisms  are  spacing,  buffer  management  and  retransmissions. 
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Spacing  The  spacing  algorithm  sets  an  upper  bound  on  the  rate  every  node  transmits  data  packets 
to  each  ot  its  neighbors.  .A  spacing  computation  follows  each  control-packet  reception  to  determine  the 
earliest  time  the  ne.xt  data  packet  can  be  transmitted  to  this  neighbor.  The  objective  is  to  provide  suthcieni 
pause  between  transmissions  to  protect  a  neighbor  node’s  processor  from  overload  and  to  avoid  collisions 
with  the  neighbor’s  transmission  of  the  packet  it  just  received. 

.\  spacing  calculation  is  illustrated  in  Figure  2.3,  where  node  .1  transmits  two  packets.  P\  and  P'_  to 
node  3.  which,  mxle  3  then  trimsmits  to  node  J.  Dotted  bo.xes  represent  packet  receptions,  solid  ho.ses 
packet  transmissions,  long  boxes  data  packets,  ard  short  bt).xes  control  packets. 


time  f|  t:  <1  tx 


1 1  .Node  3  receives  packet  1^1  from  node  3 . 

Node  A  receives  ncxle  3's  control  packet  (an  acknowledgement). 

.Node  A  has  processed  this  acknowledgement  to  the  e.xtent  that  it  must  compute  the  earliest  time  of 
Its  next  data  transmission  to  node  3  i.e.,  time  (5. 

U  Node  transmits  P:  to  node  3. 

Figure  2.3:  .A  Spacing  Computation 

When  node  3  receives  packet  P|  it  informs  node  A  (in  its  acknowledgement  packet)  of  the  transmis¬ 
sion  time  of  packet  P|  to  node  J.  Node  requires  this  information  because  P|'s  transmission  tune  by 
node  3  depends  on  node  3's  FEC  ;md  bit  rates:  Then  node  determines  the  .earliest  time  it  can  send 
packet  P:  to  node  3  by  estimating  the  time  that  node  3  completes  the  processmg  of  its  acknowledeement 
from  node  3.  This  estimation  is  done  with  the  following  formula: 

(4  =  +  P,,v.  (2.1) 

where  P  is  the  me;m  control  packet  processing  time.  This  computation  assumes  that  time  t  •.  .md 
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node  r's  transmission  of  packet  Pi  coincide,  a  reasonable  assumption  when  packets  ^ue  not  queued  for 
processing  at  either  node.  The  algorithm  in  Equation  (li  works  well  for  the  LPR. 

We've  investigated  more  sophisticated  spacing  aigonthms  but  found  they  were  not  worth  implement¬ 
ing.  For  c.xample.  we  ;maly/ed  :in  algorithm  that  required  .  ode  3  to  tell  node  A  how  long  it  would 
be  transmitting  packets  (including  moments  of  silence)  in  its  ne.xt  .sequence  of  contiguous  transmissions. 
But  this  algorithm  is  ditticult  to  implement  and  the  estimate  is  flawed  by  receptions  and  unacknow  (edged 
transmissions,  resulting  in  only  a  marginal  performance  improvement. 

Buffer  Management  The  /\’-bulfering  algorithm  limits  the  number  of  packets  that  a  node  will  buffer 
for  an\  neighbor  to  not  e.xceed  the  value  set  by  the  parameter  l\ .  When  a  node  receives  a  data  packet 
trinn  a  neighNir  for  which  K  packets  .ue  buffered  or  when  all  its  packet  buffers  have  been  allocated, 
this  packet  is  discarded  and  a  finw  -cnnn o/  packet  is  transmitted  to  inform  this  neighbor  how  long  to 
wait  before  it  may  transmit  the  packet  again.  The  first  time  a  packet  is  di.scarded  the  neighbor  is  given 
a  waiting  time  equal  to  the  elapsed  tif.ie  between  the  hrst  trimsmission  attempt  and  buffer  deallocation, 
-imoothed  over  all  data  packets  that  have  been  forwarded  for  this  neighbor.  Thereafter,  the  flow  control 
packets  contain  waiting  times  that  increase  e.xponentiaily  (until  an  upper  limit  is  reached)  until  a  buffer 
is  deallocated  or  the  packet  is  discarded.  This  flow'-control  action  reduces  packet  discarding  by  forcing 
piuticular  neighbors  to  reduce  their  transmission  rates  until  more  buffers  are  available  for  them,  thereby- 
reducing  mutual  interference  and  processing  loads. 

The  value  chosen  for  K  has  a  large  impact  on  throughput  and  fairness.  Figure  2.4  shows  a  network 
with  two.  two-hop  routes  (indicated  by  solid  arrows)  that  shtne  the  stune  tirst-hop  link  but  use  different 
second-hop  links  with  different  bandwidths.  In  general,  the  ma.ximum  end-to-end  flow  rate  for  any  route 


Figure  2.4:  The  Passing-Lane  Effect 

IS  bounded  above  by  the  speed  of  its  slowest  link,  but  interactions  between  routes  can  signiticantly 
decrea.se  the  actual  flow  rates. 

Consider  the  case  when  K  =  1.  Then  when  node  3  buffers  a  data  packet  destined  for  node  3  it 
will  not  buffer  any  more  packets  from  node  A  until  it  receives  an  acknowledgement  from  node  3  (or  it 
discards  the  packet  .U'ter  six  tries).  Thus  traffic  on  the  3  route  can  restrict  traffic  on  the  .-i  3 

route.  Now  consider  the  case  when  /\  is  larger.  Then  when  node  3  buffers  a  rlata  packet  destined  for 
node  3  It  can  buffer  another  packet  from  node  A  if  the  total  number  of  packets  buffered  for  node  .n  is 
less  than  K.  If  the  flow  rate  on  each  route  is  set  to  the  speed  of  its  slowest  link  by  the  end-to-end  flow- 
control  algonthm.  then  a  moderately  large  value  of  K  can  virtually  eliminate  the  interactions  between 
traffic  on  the  two  routes,  .md  increase  the  flow  on  the  A  3  route  to  its  ma.ximum  rate. 
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The  case  of  A’  =  I  is  analogous  to  a  one-huie  highway,  whereas  the  case  with  larger  K  is  analogous 
to  a  highway  with  passing  lanes.  A  small  value  of  K  not  only  lowers  the  throughput  of  some  routes,  it 
also  biases  the  measurements  given  to  the  end-to-end  tlow-control  algonthm  and  results  in  unlair  end- 
to-end  flow  allocations.  To  avoid  these  adverse  interactions  between  the  linJc-level  congestion-avoidance 
algorithms  and  the  end-to-end  flow  control  algorithm  we  advocate  the  use  of  a  large  K  value,  even 
though  this  can  result  in  a  node  ocassionally  allocating  a  disproportionate  number  of  buffers  to  a  few 
neighbors  for  short  time  periods.  The  LPR  has  eleven  packet  buffers;  we  have  set  A'  to  seven  ;uid  to 
prevent  deadlocks,  we  attempt  to  keep  iwo  hutfers  in  reserve  at  all  times  for  receiving  packets. 

Retransmissions  The  retransmission  algorithm  is  used  by  a  node  to  set  the  transmission  times  to  a 
neighbor  that  is  not  retummg  control  packets,  either  because  it  is  blocking  data  packet  transmissions  or 
because  it  cannot  decode  the  part  of  the  packets  that  contains  the  sender  ID  and  remrn  a  gam  adjustment 
packet.  The  (contlictmg)  objectives  of  the  algorithm  are  fourfold; 

1.  increase  throughput  by  retransmitting  quickly 

1.  maintam  channel  stability  by  retransmittmg  slowly 

decrease  the  blocking  rate  by  staggering  transmissions  when  more  than  one  node  is  retransmitting 
to  the  same  node 

4.  prevent  retran.smissions  from  occuring  prematurely,  i.e..  before  control  packets  can  be  nansmitted. 

If  the  first  item  were  not  a  concern,  the  design  of  the  retransmission  algorithm  would  he  quite  simple. 
In  this  case  we  would  use  a  random  backoff  algorithm  with  a  large  average  delay,  e.g.,  an  algorii  im 
that  chooses  retran.smission  times  so  their  values  are  uniformly  distributed  on  time  intervals  that  mcrease 
rapidly  in  length  with  the  number  of  consecutive  unacknowledged  retransmissions.  But  because  block¬ 
ing  occurs  frequently,  this  conservative  approach  mtroduces  too  much  delay  and  consequently  reduces 
throughput.  Instead,  we  have  taken  the  following  more  aggressive  approach. 

The  retransmission  algonthm  inserts  sufficient  pau.se  between  retransmissions  to  make  the  event  that 
a  retransmission  occurs  before  a  control  packet  can  be  transmitted  extremely  rare.  It  achieves  this  goal 
by  using  a  constant  bias  equal  to  The  first  retransmission  time  increment  is  given  by: 

T\  =  r ,,  iif  +■  Di 

where  {  D, }  is  a  sequence  of  independent  random  vanables  that  are  uniformly  distributed  over  a  time 
interval  given  by  the  sum  of  a  Moment  of  Silence  and  the  maximum  packet  transmission  time.  With 
each  subsequent  retry  the  intertransmission  time  increment  increases  linearly  m  mean: 

T’-i+i  =  ii:  for  n  =  1.2 . 5 

We  call  the  resulting  algonthm  linear  backoff. 

When  a  gain  adjustment  packet  is  received  a  retransmission  is  also  needed,  but  the  requirements 
for  scheduling  this  retransmission  are  quite  different  becau.se  the  cause  is  channel  noise,  not  blocking. 
In  these  cases  the  link  gain  is  adjusted  appropnately  and  the  packet  is  retransmitted  after  a  very  short 
random  delay. 
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2.2.3  Congestion  Avoidance  (Algorithms  from  SCRAP) 

The  remaining  three  algorithms  are  SURAP  algorithms  that  we  have  used  in  our  CDMA  network  archi¬ 
tecture. 


Fairness  Queueing  The  LPR  has  a  single  Transmit  (^eue  that  contains  packets  waiting  to  be  trans¬ 
mitted.  Packets  are  processed  FIFO  as  they  are  received  and  inserted  in  this  queue;  the  transmissions 
however  are  not  necessarily  FIFO.  There  are  two  exceptions. 

First,  a  packet  is  never  transmitted  until  its  transmission  timer  expires.  This  timer  is  set  when  a  packet 
is  first  processed  and  reset  with  each  transmission.  Because  the  pacing  (used  only  in  SURAP,  see  Section 
2.3.2)  and  spacing  (used  only  in  CDMA,  see  Section  2.2.2)  algorithms  can  set  different  transmission  rates 
to  different  neighbors  a  packet  going  out  a  “fast”  link  can  be  transmitted  before  a  packet  going  out  a 
“slow”  link,  even  when  the  packet  going  out  the  slow  link  arrived  first.  This  feature  enhances  the  passing 
lane  effect  illustrated  in  Figure  2.4. 

Second,  link  reliability  can  vary  greatly  in  a  packet  radio  network;  with  straight  FIFO  queueing,  the 
more  reliable  links  could  monopolize  buffer  and  channel  resources,  creating  unfairness.  When  packets 
received  from  more  than  one  neighbor  are  queued  with  the  same  “next-hop”  destination, /airnerj  queueing 
will  not  allow  consecutive  packet  transmissions  from  the  same  neighbor  to  this  destination.  In  these 
cases  the  order  of  the  packet  transmissions  is  modified  so  packets  will  be  transmitted,  round  robin,  on  a 
neighbor-by-neighbor  basis.  This  prevents  a  neighbor  with  a  reliable  link,  or  a  host  with  a  wire  interface 
from  getting  an  unfair  bandwidth  allocation. 


Alternate  Routing  After  a  packet  has  been  transmitted  four  times  without  an  acknowledgement  a 
special  “alternate-routing”  flag  is  set  in  the  packet  header  and  it  is  broadcast  on  the  last  two  transmissions 
to  request  help  in  forwarding  the  packet.  Normally  packet  radios  will  never  transmit  received  data  packets 
that  are  not  addressed  to  them.  But  when  the  alternate-routing  flag  is  set  and  the  destination  is  listed 
in  their  routing  tables  with  a  shorter  distance  they  attempt  to  forward  the  packet.  This  feature  allows 
packets  to  be  quickly  rerouted  around  temporary  link  failures.  If  the  problem  persists,  the  packet  radios 
will  update  their  routing  tables  to  eliminate  the  link. 

When  nodes  in  a  CDMA  network  hear  an  alternate-routed  packet,  they  can  bombard  the  transmitter 
with  overlapping  acknowledgement  transmissions  and  the  mutual  interference  could  prevent  a  correct 
reception  of  any  of  them.  We’ve  modified  the  SURAP  alternate-routing  algorithm  for  CDMA  to  avoid 
this  flood  by  randomizing  the  transmission  time  of  acknowledgements  to  alternate-routed  packets.  The 
randomization  interval  is  of  the  order  of  a  Moment  of  Silence. 


Packet  Dropping  Both  protocols  set  a  limit  of  six  on  the  number  of  times  a  node  can  transmit  a  data 
packet  without  receiving  an  acknowledgement.  If  on  the  sixth  transmission  attempt  no  acknowledgement 
is  received,  the  packet  is  discarded  and  higher-level  algorithms  may  resubmit  it  to  the  network  at  a 
later  time.  Packet  dropping  is  an  important  mechanism  for  purging  a  network  of  undehverable  traffic  to 
prevent  deadlock  when  it  is  congested. 
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2.2.4  Routing 

Routes  are  calculated  in  the  LPRnet  with  a  distributed  minimum-hop  algorithm,  called  tier  routing.  The 
tier  level  of  a  destination  is  the  number  of  hops  required  to  reach  it.  This  algorithm  requires  each  node 
to  maintain  a  tier  table  that  lists  which  packet  radio  is  the  next  hop  enroute  to  each  reachable  destination. 
Each  LPR  broadcasts  its  tier  table  at  an  average  rate  of  once  per  7.5  seconds  in  a  special  packet  called  a 
PROP.  As  packet  radios  receive  PROPs  they  learn  of  shorter  routes  to  old  destinations  and  routes  to  new 
destinations  and  update  their  tier  tables  accordingly. 

2.3  Broadcast  Networks 

We  describe  the  link-layer  protocol  used  in  SURAP.  Variations  of  this  design  are  used  by  SURAN, 
SINCGARS  [5]  and  an  experimental  system  for  library  automation  [10].  A  detailed  description  of 
SURAP  is  published  in  a  paper  by  Jubin  and  Tomow  [2],  however  the  reader  must  be  aware  that  the 
current  SURAP  algorithms  deviate  slightly  from  this  reference.  The  only  noteworthy  impact  on  our 
performance  comparison  was  the  elimination  of  “quadratic  backoff  which  has  been  demonstrated  to 
cause  unfair  end-to-end  flow  allocations[li].  This  and  other  less  significant  modifications  to  SURAP 
have  been  incorporated  into  our  broadcast-network  simulation.  In  spite  of  these  changes  [2]  is  still 
an  excellent  introduction  to  broadcast  link-layer  design  and  other  related  packet-radio  algorithms,  e.g., 
routing. 

2.3.1  Link-Level  .Acknowledgements 

A  fundamental  concept  in  broadcast  link-layer  design  is  the  passive  acknowledgement.  A  broadcast 
channel  allows  any  node  in  range  of  a  packet  transmission  to  receive  it.  For  example,  when  node  5 
(Figure  2.5)  transmits  in  an  attempt  to  forward  a  data  packet  to  node  C.  both  nodes  A  and  C  can  receive 
this  transmission.  Node  C  will  process  the  packet  and  prepare  it  for  transmission  to  node  P,  whereas 
node  A  will  recognizes  the  packet  as  an  earlier  transmission  to  node  B .  To  node  C  it  is  a  data  packet 
reception;  to  node  >1  it  is  a  passive  acknowledgement.  A  single  transmission  serves  both  purposes. 

Note  that  there  are  two  cases  when  passive  acknowledgements  carmot  be  used.  A  packet  radio  (e.g. 
node  P)  will  not  use  its  radio  transmitter  to  forward  packets  it  receives  as  the  final  destination  of  a  route. 
Instead  it  will  send  them  over  a  wire  interface  to  an  attached  host.  In  these  cases  a  separate,  or  active 
acknowledgement  transmission  is  required.  This  is  illustrated  in  Figure  (2.5)  by  node  P  nansmitting 
an  active  acknowledgement  packet.  The  other  situation  that  requires  an  active  acknowledgement  arises 
when  a  passive  acknowledgement  is  blocked  or  corrupted  with  bit  errors.  If,  for  example,  node  A  in 
Figure  (2.5)  does  not  receive  node  3 ’s  passive  acknowledgement,  node  A  will  repeat  the  data  transmission 
and  node  B  will  respond  with  an  active  acknowledgement  if  it  receives  the  packet. 

2.3.2  Congestion  Avoidance 

Single  Threading  The  single-threading  policy  outlined  by  Jubin  and  Tomow  is  equivalent  to  a  link- 
level  window  size  of  one,  meaning  a  policy  of  not  transmitting  a  new  packet  to  a  neighbor  until  the 
most  recent  transmission  has  been  acknowledged.  When  single  threading  is  combined  with  passive 
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Figure  2.5:  An  Illustration  of  Passive  Acknowledgements  and  Pacing 


acknowledgements,  the  resulting  policy  implicitly  limits  the  number  of  packets  from  each  neighbor  that 
a  packet  radio  will  buffer.  For  example,  suppose  that  node  A  in  Figure  (2.5)  transmits  a  data  packet  to 
node  B  and  the  packet's  next  destination  after  8  is  C.  Here  5  will  not  acknowledge  this  packet  —  or 
equivalently,  will  not  transmit  the  packet  to  C  —  until  any  previous  packets  it  has  transmitted  to  C  are 
acknowledged.  Hence,  the  combined  policy  tends  to  have  radios  buffer  only  one  packet  for  each  radio 
forwarding  through  it.  The  objective  of  single  threading  is  to  limit  the  number  of  packets  in  the  subnet 
and  to  create  “backpressure”  which  prevents  packets  from  entering  the  network  until  it  has  adequate  free 
resources. 


Pacing  The  Pacing  algorithm  sets  packet  inter-transmission  times  on  a  neighbor-by-neighbor  basis. 
Whenever  a  packet  radio  transmits  a  particular  data  packet  it  records  the  time  the  transmission  ends. 
When  an  acknowledgement  is  received  for  this  transmission,  the  time  the  acknowledgement  reception 
concludes  is  also  recorded.  The  difference  between  the  two  times  (labeled  0  in  Figure  (2.5»  is  called 
the  forwarding  delay.  It  includes  processing,  queueing  and  transmission  delays  at  the  receiving  node. 
Forwarding  times  are  measured  and  smoothed  by  each  node  on  a  neighbor-by-neighbor  basis  to  maintain 
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a  short-term  history  of  the  delay  through  each  neighbor. 

The  pacing  delay  (labeled  Pacing  in  Figure  (2.5)).  on  the  other  hand,  determines  the  time  between 
successive  data  transmissions  to  each  neighbor.  Its  defined  as  three  times  the  forwarding  delay  since  the 
receiving  node  must  receive,  transmit  and  receive  an  acknowledgement  for  each  packet[3].  The  dashed 
line  indicates  the  earliest  time  node  A  can  transmit;  because  of  forwarding  delay  variance  the  pacing 
delay  will  occassionally  overestimate  or  underestimate  the  minimum  inter-transmission  time. 

2.4  Architecture  Coinpari.son 

In  this  Section  we  summarize  the  differences  between  the  SURAP  and  the  CDMA  protocol  suites  and 
introduce  the  performance  issues  addressed  in  Section  3. 

2.4.1  Blocking  and  Mutual  Interference 

As  shown  in  [8],  CDMA  networks  can  generally  sustain  a  higher  throughput  because  they  allow  concur¬ 
rent  packet  transmissions  to  different  destinations  in  the  same  radio  neighborhood.  However,  for  some 
network  configurations  the  mutual  interference  caused  by  these  additional  transmissions  can  increase  the 
bit  error  rate  and  decrease  throughput.  Broadcast  networks,  on  the  other  hand,  tend  to  prohibit  concurrent 
transmissions  because  whenever  a  packet  radio  transmits,  every  idle  packet  radio  in  radio  range  locks 
onto  the  transmission,  thus  preventing  all  from  transmitting  until  the  transmission  is  over.  This  staggering 
of  transmissions  reduces  mutual  interference.  The  mumal  interference  difference  is  most  profound  when 
the  network  has  no  hidden  terminals,  and  connectivity  is  high,  e.g.  Figure  3.6. 

The  reduced  mutual  interference  in  broadcast  networks  tends  to  increase  throughput  because  the 
lower  bit  error  rates  decrease  the  retransmission  rate.  However  the  transmission  staggering  in  broadcast 
networks  decreases  the  rate  the  individual  radios  can  transmit  data  packets  and  thus  decreases  throughput. 
Also  packet  radios  in  a  broadcast  network  can  block  transmissions  when  they  receive  “unwanted  packets”; 
this  effect  is  quite  noticable  when  there  are  hidden  terminals,  e.g..  Figure  3.3. 

2.4.2  Acknowledgement  Traffic  and  Power  Control 

In  the  absence  of  blocking  and  mutual  interference,  an  iV-hop  route  in  a  broadcast  network  requires 
iV  -I- 1  transmissions:  N  data  packet  transmissions  and  one  active  acknowledgement  transmission.  The 
same  route  in  a  CDMA  network  requires  2  x  N  transmissions:  N  data  transmissions  and  .V  active 
acknowledgement  transmissions.  For  example,  in  Figure  2.5  three  data  packet  transmissions  and  one 
active  acknowledgement  transmission  are  needed  to  forward  one  data  packet  along  a  three  hop  route. 
The  same  route  in  a  CDMA  network  would  require  three  data  packet  and  three  active  acknowledgement 
packet  transmissions.  Based  on  transmission  counts  alone,  it  seems  broadcast  networks  have  a  more 
efficient  chaimel  that  can  allocate  more  bandwidth  to  data  packet  transmissions  because  less  bandwidth 
is  required  for  acknowledgements. 

This  efficiency  argument  is  valid  when  there  is  no  blocking  and  power  control  is  not  considered  but 
it  is  not  true  in  general.  When  a  broadcast  channel  sends  a  data  packet  and  a  passive  acknowledgement 
with  the  same  transmission,  the  power  levels  needed  to  reach  the  two  nodes  can  differ  by  several  orders 
of  magnitude.  In  general,  whenever  passive  acknowledgements  are  used  on  a  broadcast  chaimel  each 
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packet  must  be  transmitted  at  the  maximum  of  the  two  power  levels.  When  the  acknowledgement  power 
requirement  is  higher,  a  shorter  active  acknowledgement  transmission  at  this  power  level  uses  less  energy 
and  thus  reduces  mutual  interference.  For  some  configurations  higher  throughputs  could  be  achieved  with 
a  broadcast  chaimel  if  active  acknowledgements  were  used. 

2.4.3  Transmission  Scheduling  and  Flow  Control 

In  a  broadcast  network  the  passive  acknowledgement  transmissions  must  be  scheduled  with  the  same 
algorithm  as  data  packet  transmissions  because  one  transmission  serves  two  purposes.  The  scheduling 
objectives  for  data  transmissions  and  acknowledgment  transmissions  are  in  conflict.  Data  packet  trans¬ 
missions  must  be  scheduled  to  achieve  flow  control,  channel  stability,  etc.,  whereas  acknowledgement 
transmissions  should  be  scheduled  to  free  buffers  quickly.  Bundling  the  two  transmissions  together 
with  the  use  of  passive  acknowledgements  necessitates  a  performance  compromise.  The  SURAP  Pacing 
algorithm  is  designed  for  congestion  avoidance,  not  for  quick  reliable  acknowledgement  transmissions. 

The  CDMA  network,  on  the  other  hand,  transmits  separate  acknowledgement  and  data  packets.  The 
Moment  of  Silence  algorithm  keeps  the  acknowledgement  blocking  levels  very  low  and  the  Spacing 
algorithm  schedules  data  packet  transmissions  to  reduce  blocking  while  expediting  the  flow  of  data 
packets.  This  twofold  strategy  significantly  improves  throughput  by  decreasing  data  and  acknowledgement 
packet  blocking. 

The  Pacing  algorithm  used  by  SURAP  determines  the  time  between  a  node’s  transmissions  to  each 
of  its  neighbors  by  measuring  their  forwarding  delays.  But  because  of  changes  in  link  reliability  and 
packet  transmission  times  these  statistics  can  have  significant  variance,  resulting  in  the  Pacing  algorithm 
over-  and  under-estimating  pacing  delay.  This  increases  blocking  and  decreases  throughput. 

The  Spacing  algorithm,  on  the  other  hand  does  not  require  a  node  to  measure  the  duration  of  its 
neighbors  transmissions,  instead  each  successful  transmission  is  answered  with  a  control  packet  that 
specifies  the  duration  of  the  next  transmission  of  the  packet  on  the  next  leg  of  its  Journey.  This  up-to- 
date  information  allows  a  node  to  schedule  its  transmissions  more  accurately,  thereby  reducing  blocking. 

2.4.4  Fairness 

SURAP  sets  end-to-end  flow  rates  with  a  combination  of  Pacing  and  Single-Threading.  This  strategy 
allows  each  packet  radio  to  set  the  flow  rates  on  each  of  its  links  unilaterally  which  can  result  in  unfair 
end-to-end  flow  rates. 

The  CDMA  protocol  suite,  on  the  other  hand,  employs  two  cooperating  congestion-avoidance  mech¬ 
anisms,  an  end-to-end  flow-control  algorithm  that  establishes  long-term  —  relative  to  the  length  of  a  few 
packet  transmissions  —  max/min  fair  end-to-end  flow  rates,  and  link-level  congestion-avoidance  algo¬ 
rithms,  e.g.,  if -buffering  and  Spacing  that  facilitate  the  flow  of  data  packets  and  provide  “passing  lanes”, 
protect  critical  nodal  resources  from  overload  during  traffic  bursts,  and  provide  a  “safety  net”  to  prevent 
the  spread  of  congestion  when  the  end-to-end  Sow-control  algorithm  is  adapting  to  a  change  in  traffic 
patterns. 
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3.  Performance  Analysis 


We  present  simulation  results  that  compare  the  performance  of  SURAP  and  the  CDMA  protocol  suites. 
Because  packet-radio  network  performance  is  a  complex  fimction  of  topology,  node  proximity,  and 
network  traffic  patterns  it  is  infeasible  to  investigate  all  these  effects  in  networks  of  significant  size  by 
simulation.  Therefore,  we  have  taken  the  approach  of  identifying  representative  “topological  components” 
of  packet-radio  networks,  e.g.,  splits,  merges,  and  parallel  flows  with  radio  interference,  that  test  a 
network’s  throughput  and  fairness  while  maintaining  a  feasible  number  of  nodes.  We  ran  extensive 
simulations  with  these  components  to  improve  our  design  and  tune  its  parameters.  After  we  completed 
this  exploratory  work,  we  repeated  the  simulations  with  the  SURAP  algorithms  to  establish  a  comparison 
between  the  two  network  architectures.  We  report  some  surprising  and  dramatic  performance  differences. 

3.1  Background  Information 

Our  results  are  from  a  discrete-event  simulation  that  models  radio  interference.  As  a  result  were  able  to 
invesigate  not  only  the  network  performance  impact  from  link-level  algorithms  but  the  affects  of  radio 
interference  as  well. 

The  radio  interference  model  computes  the  signal-tq-noise  ratio  at  each  receiver  for  each  of  its  packet 
receptions,  by  including  the  affects  of  antenna  gain,  processing  gain,  and  path  loss.  The  resulting  signal- 
to-noise  ratio  is  mapped  into  a  bit-error  rate  by  the  use  of  the  Gaussian  Q  function  then  this  rate  is  mapped 
into  the  number  of  bit  errors  in  each  packet  by  an  efficient  Monte-Carlo  technique.  After  the  number  of 
bit  errors  have  been  computed,  a  heuristic  approach  is  used  to  compute  the  probability  that  the  errors  can 
be  corrected  by  the  convolutional  FEC  decoder.  This  model  is  discussed  in  detail  in  reference[7]. 

The  discrete  event  simulation  assumes  heavy  traffic,  i.e.,  all  route  sources  send  traffic  at  the  highest 
rate  allowed  by  the  network  protocols.  Packet  lengths  are  independent  and  uniformly  distributed  between 
528  and  2428  bits,  including  headers,  for  a  mean  packet  length  of  approximately  1500  bits. 

The  LPR  has  two  processors,  one  that  operates  the  transceiver  and  one  for  processing  packets.  We 
model  processing  times  in  the  following  way.  There  is  a  one  millisecond  delay  in  the  LPR  when 
the  transceiver  gives  a  packet  to  the  packet  processor.  The  packet  processor  takes  a  minimum  of  five 
milliseconds  to  process  a  packet  to  be  sent  over  the  wire  interface  and  a  minimum  of  eight  milliseconds  if 
the  packet  is  to  be  transmitted  on  the  radio  channel.  Either  of  these  two  processing  tasks  can  take  longer 
when  interrupts  occur  or  transmission  timers  expire.  Each  of  these  events  takes  two  milliseconds  and  on 
average  one  occurs  per  each  received  packet.  Thus,  the  minimum  processing  time  for  a  received  packet 
that  is  sent  over  the  wire  interface  is  six  milliseconds  and  the  minimum  processing  time  otherwise  is 
nine  milliseconds.  Average  processing  times  for  the  two  cases  are  roughly  eight  and  eleven  milliseconds, 
respectively.  Acknowledgements  take  the  same  amount  of  time  to  process  as  data  packets  that  are  sent 
over  the  wire  interface. 
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We  have  organized  our  simulation  results  into  four  catagories:  Blocking  and  Mutual  Interference, 
Jamming,  Acknowledgement  Effects,  Fairness,  and  Fault  Recovery.  The  first  three  categories  contain 
experiments  designed  to  investigate  physical  and  link-level  issues.  The  latter  two  categories  are  concerned 
with  higher-level  or  systematic  issues.  These  results  are  presented  in  the  following  ten  subsections.  Each 
one  has  a  separate  panel  that  contains  a  figures  depicting  the  network  topology  and  plot  of  simulation 
results.  The  convention  for  showing  network  topology  is  that  dotted  lines  show  radio  connectivity  and 
solid,  arrowed  lines  show  routes.  Each  panel  is  followed  by  a  brief  experimental  description. 

Each  simulation  result  shows  throughput  as  a  function  of  time.  We  have  rerun  each  simulation  with 
several  random-number-generator  “seed  values”  and  we  have  determined  that  the  seed  value  used  for 
generating  a  throughput  curve  does  not  have  a  significant  impact  on  our  results. 
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3.2  Blocking,  and  Mutual  Interference 
3.2. 1  Parallel  Flows 

50 
45 
40 
35 
30 
25 
20 
15 
10 
5 
0 

0  50  100  150  200  250  300 

Time  (seconds) 

Figure  3.1:  Aggregate  Throughput 
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Figure  3.2:  Route-by-Route  Throughput 


This  experiment  was  designed  to  show  the  effect  of  Code  Division  on  network  throughput.  In  order  to 
simplify  the  network  diagram  we  lumped  nodes  into  clusters.  For  example  nodes  A,  B,  and  C  are  in  the 
same  cluster.  Every  node  is  in  radio  connectivity  with  each  node  in  its  cluster  and  each  node  in  every 
adjacent  cluster. 


18 


Report  No.  7173 


BBN  Systems  and  Technologies  Corporation 


Figure  3.1  shows  the  aggregate  throughput  for  th.*  two  protocols  suites.  Note  that  the  CDMA  al¬ 
gorithms  deliver  more  than  twice  the  throughput  of  SCRAP.  If  the  three  routes  had  disjoint  paths  the 
throughput  improvement  would  be  approximately  three-fold  but  because  of  interactions  between  routes 
the  throughput  is  lower.  Notice  that  between  125  and  225  seconds  into  the  simulation  the  C  =>  C  route 
approximately  doubles  its  throughput,  while  the  other  two  roughly  halve  their  throughput.  On  closer 
examination,  we  determined  that  during  this  time  interval  the  A  =>  J  and  the  B  =>  K  route  intersect 
at  node  D ,  while  the  other  route  follows  a  disjoint  path.  Nevertheless,  this  bandwidth  allocation  is  fair, 
given  the  particular  (suboptimal)  routing. 
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3.2.2  Hidden  Terminals  (Part  I) 
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Fig\ire  3.3:  A  ^  9  Throughput 


This  simulation  demonstrates  how  hidden  terminals  can  reduce  network  throughput.  Nodes  A,  B,  and 
C  are  hidden  from  nodes  D ,  £ ,  and  /.  Node  $  on  the  other  hand  can  hear  every  node  in  the  network. 
Our  simulation  showw  that  the  CDMA  version  of  this  network  has  about  four  times  the  throughput  of 
the  SURAP  version.  The  throughput  reduction  with  SURAP  is  because  of  data  and  acknwowledgement 
packet  blocking.  We  anticipate  this  network  configuration  to  occur  frequently,  especially  when  repeaters 
are  placed  on  the  tops  of  hills  to  connect  hidden  radios. 
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3.2.3  lliddcn  rerniinals  (Part  Ul 
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Figure  3.4.  Aggregate  Throughput 
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Figure  3.5:  Route-by-Rouie  Throughput 


This  simulation  illustrates  problems  that  iKCur  when  a  single  mxle  listens  to  many  hidden  terminals. 
The  central  node  can  overhear  the  irattic  trom  all  radios  many  of  which  cannot  hear  each  c>ther;  with 
SCRAP  this  causes  it  to  hliKk  packets  at  a  very  high  rate.  This  is  why  the  ~  =^->  S  and  2  7  routes 

achueve  such  tow  throughputs  with  SCR.AP. 

The  central  node  stops  operating  one  hundred  seconds  into  the  simulation.  This  reduces  the  CDMA 
network  throughput  but  has  no  significant  affect  on  the  SCRAP  throughput,  indicating  that  the  CDMA 
network  is  more  effectently  utilizing  its  available  resources.  Or.  adding  resources  to  a  SUR-AP  network 
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3.3  lamming 

3.3.1  High  Conncctixily 
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Figure  3.6:  Jamming  Effect  on  .-i  3  Throughput 


We  simulated  ten  packet  radio  nodes  in  mutual  radio  connectivity  ,  NcxJe  .4  is  attacked  by  a  jammer.  The 
jamming  power  increases  Imearlv  over  the  duration  of  the  experiment,  starting  at  .5  picowatts  and  ending 
at  .62  picowatts. 

When  the  jamming  power  is  low  the  CDM.A  network  has  about  three  times  the  throughput  of  the 
SCRAP  network,  because  the  former  allows  simulumeous  packet  transmissions  to  different  desttnations. 
For  this  particular  e.xample  the  SCR.\P  network  has  no  packet  blocking  because  every  packet  transmission 
IS  received  by  every  packet  radio,  preventing  them  from  transmitting  at  the  same  time.  This  staggering 
reduces  mutual  interference. 

,\s  the  jammmg  pow,er  increases,  the  throughput  of  the  CDM.'X  network  decreases  because  the  mutual 
interference  and  jamming  reduce  the  signal-to-noise  ratio.  This  demonstrates  that  a  SCRAP  network  can 
be  more  survivable  than  a  CDMA  network  when  there  are  no  hidden  terminals  and  when  there  is  a  large 
amount  of  jamming  noise. 
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3.3.2  Blofkino 
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Figure  3.7:  Jamming  Etfect  on  A  >  r  Throughput 


This  simulation  illusu-ates  the  etfect  of  jamming,  hidden  terminals,  tuid  mutual  interference.  Nodes  2. 

.  and  .*  transmit  packets  as  fast  as  possible,  whereas  the  intertransmission  times  of  nodes  T  and  Jure 
independent  and  expiinentially  distributed  '•cith  mean  300  milliseconds. 

Note  that  node  2  and  2  must  deal  with  hidden  terminals.  For  example  when  node  T  transmits  to 
node  S  It  IS  possible  that  ncxle  :  will  not  hear  the  packet  because  it  is  listening  to  node  2's  transmis¬ 
sion.  Therfore.  tuxle  T  must  retransmit,  causing  additional  blocking  and  mutual  interference  for  packets 
transmitted  from  node  A  to  node  f . 

Even  as  the  jamming  at  node  3  increases  the  throughput  of  the  CDM.\  network  is  greater  than  the 
SCRAP  network.  Thus  CDMA  networks  c;m  be  more  survivable  when  there  tue  many  hidden  terminals 
present. 
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3.4  Acknowledtiement  Tratlic  F.fiects 
3.4.1  Mutual  Interference 
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Figure  3.8:  Route  T  i  Throughput 

This  simulation  shows  the  benetit  of  transmittmg  short  active  acknowledgements  instead  of  longer  passive 
acknowledgements.  The  power  level  used  by  node  3  when  it  transmits  to  node  .-i  determines  the 
amount  of  mutual  itnerference  these  transmissions  create  for  node  3  's  receptions  from  node  3.  During 
the  e.xpenment  the  link -gain-adjustment  algonthm's  settmg  for  node  3 's  transmissions  to  node  .-i  were 
overridden  and  set  to  either  the  maximum  or  minimum  possible  power  level  to  illustrate  this  effect. 
Notice  that  SCRAP  is  quite  sensitive  to  the  power  used  by  acknowledgements;  the  throughput  on  the 
3  =>  f  route  decreases  about  25^  when  the  acknowledgements  are  transmitted  at  mnximum  power. 
The  CDMA  network,  on  the  other  hand,  appears  to  be  msensitive  to  these  power  levels.  The  diffc:cnce 
is  due  to  the  relative  sizes  of  the  two  kmds  of  acknowledgements;  the  passive  acknowledgement  is  the 
size  of  a  data  packet  and  consequently  generates  much  more  mutual  interference  thtui  the  much  smaller 
active  acknowledgement. 
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3.4.2  Acknouledgenient  Priority  and  Fairness 
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Figure  3.9:  Route-by-Route  Throughput 


This  simulation  demonstrates  the  effect  of  promptly  transmitting  acknowledgements.  In  the  SL’R.\P 
e.xpenment  there  is  roughly  a  three-fold  difference  m  throughput  between  the  two  routes,  but  in  the 
CDMA  e.xpenment  the  throughputs  are  almost  identical  (after  a  brief  warm-up  period). 

In  the  SUR.AP  experiment  node  3  does  not  acknowledge  a  reception  from  node  .s' until  it  transmits 
the  packet  to  node  3.  But  node  3  is  also  the  source  of  a  route  to  node  3.  thus  there  is  no  guarantee  that 
node  3  will  promptly  transmit  any  packet  it  has  received  from  node  .4;  instead  it  may  be  attempting  to 
forward  its  own  packet  to  node  3.  This  effect  causes  the  Pacing  algorithm  to  slow  down  transmissions 
from  node  A.  Because  the  CDMA  network  expedites  acknowledgement  transmissions  this  effect  cannot 
pe 
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3.5  Fairness 

3.5.1  The  Passin'4T,ane  Effect 
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Figure  3.10:  Route-by-Route  Throughput 

This  experiment  demonstrates  the  “passing  lane"  effect.  Nodes  A.  i.  and  i  all  have  routes  to  node 
'  that  must  flow  through  node  5.  The  heavy  flow  of  traffic  through  node  T  makes  the  r-to-f  link  have 
tar  le.ss  capacity  than  the  r-to-,”link.  When  the  network  runs  SUR.\P  the  congestion  at  node  5  spreads 
to  the  5 -to-.”  route,  but  when  the  network  runs  the  CDMA  algorithms  node  T  provides  a  passing  lane 
tor  the  5 -to-.”  traffic  and  prevents  the  spread  of  congestion. 
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3.5.2  .Splits  and  Merges 


\ 


CDMA  SCRAP 


25 

- ^ ^ ^ ^ ^ - 1  25 

- - - 1 - ! - 1 - 1 - 1 - r- 

20 

! 

^  20 

i 

_ 

A  N  to  i.'  1 

A  \  to  2 

15 

^  /  to  J  15 

A  /  to  ■ 

Pkts/Sec. 

3  to  4  1 

5  to  ^ 

10 

-  ♦  10 

to  2  _ 

1 

i 

1 

A  A-  A-  4'  4'  4 

5 

"A-  f  *  4.  #  *  1  5 

4  A  A  4  A  A  A 

4  V  V 

0 

- ^ ^ ^ ^ - 1  0 

Q  I _ I _ 1 _ ^ ^ _ 1 _ 1  ()  _ I _ ^ _ ! _ I _ i _ ^ _ I _ ! 

0  20  40  60  SO  100  120  140  0  20  40  60  80  100  120  140 


Time  (seconds)  Time  (seconds) 


Figure  3.11;  Route-by-Route  Throughput 

This  experunent  demonstrates  that  the  end-to-end  flow-control  algonthm  in  the  CDMA  network  assigns 
ma.x/mm  fair  end-to-end  flow  rates,  but  the  SCRAP  network  does  not.  For  this  particular  topology  all 
trafrtc  must  pass  through  a  single  node  (node  S),  thus  max/min  fairness  in  this  case  is  the  same  as 
equality. 
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3.6  Fault  Recosery 


CDMA 


SCRAP 


25 


20  - 


I - r 


to  -2 
3  to  - 


15  [- 


Pkts/Sec. 


10 


3  “ 


J _ l_ 


J 


25 


20  - 

15  - 

-  10  k 

K 

H  5  i- 

! 


1 - 1 - 

-i c 

3  to  w 


^  *  iT  ♦  •  ♦  *  ^ 


0 


0  100  200  ?00  400  500  600  0  100  200  300  400  500  600 

Time  ( seconds)  Time  (seconds) 


Figure  3.12:  Route-by-Route  Throughput 


This  experiment  shows  an  example  where  a  node  failure  spreads  congestion  when  the  network  is  rurming 
SCRAP  but  not  when  when  it  runs  the  CDMA  algorithms.  Node  d  is  powered  down  300  seconds  into 
the  simulation.  SCRAP  causes  the  .-i-to-.”  throughput  to  drop  for  a  few  seconds.  This  effect  has  never 
been  detected  with  the  CDMA  network. 


29 


BBN  Systems  and  Technologies  Corporation 


Report  No.  7173 


4.  ('inulusions 


We  have  demonstrated  that  tor  many  network  topologies  the  CDMA  algorithms  produce  greater  throughput 
than  the  SCRAP  algonthms.  Also,  the  CDMA  algorithms  provide  the  additional  benetit  of  max, min-t'air 
bimdwidth  allocation.  But,  becuase  the  performance  ot  a  packet-radio  network  is  so  strongly  dependent 
on  network  topologv,  we  c:mnot  say  in  general  that  the  CDMA  algorithms  are  superior  to  the  SL  ICAP 
algorithms.  Nevertheless  we  beleive  that  the  CDMA  algorithms  are  superior  to  the  SCRAP  algorithms 
tor  most  network  topologies  and  when  this  is  the  ca.se  the  pert'ormance  difference  is  quite  remaikable. 

The  most  obvious  limitation  of  the  CDMA  algonthms  is  demonstrated  by  Figure  .Cb.  This  e.\am- 
ple  shows  that  when  a  packet-radio  network  has  no  hidden  terminals,  mutual  interference  coupled  vvith 
jamming  can  render  the  SUR.\P  algonthms  superior  to  the  CDM.A.  algonthms.  We  beleive  that  addi¬ 
tional  research  is  needed  in  this  area.  Perhaps  a  mechanism  can  be  developed  that  will  allow  a  packet 
radio  to  detect  the  difference  between  jamming  imd  mutual  interference  and  adjust  its  power  levels  and 
transmission  rate  accordingly.  Perhaps  this  result  would  make  CDMA  algorithms  superior  to  the  SCRAP 
algonthms  for  all  network  topologies. 
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Appendix  A.  End-to-End  Flow  Control 


This  appendix  provides  a  brief  overview  of  the  end-to-end  flow-control  algorithm  used  by  our  CDMA 
packet  radio  network.  We  demonstrate  the  need  for  end-to-end  flow  control  with  a  simple  example  that 
shows  how  the  pacing  algorithm  in  SURAP  causes  unfair  end-to-end  flow  allocations.  Consider  the 
following  example  network  with  three  routes  (indicated  by  dotted  lines): 

1.  .4  =>  7 

2.  3  =>  7 

3.  f  =>  7. 


Figure  A.l:  Unfairness  Caused  by  Pacing 

SURAP  uses  no  explicit  end-to-end  flow-control  mechanism  to  allocate  bandwidth  to  routes.  Instead, 
end-to-end  flow  rates  are  an  implicit  function  of  the  link  flow  rates  set  by  the  pacing  algorithm,  the 
single-threading  policy,  and  fairness  queueing.  This  design  results  in  fair  bandwidth  allocations  to  the 
inboimd  flows  at  each  packet-radio  but  it  causes  unfair  bandwidth  allocation  to  routes. 

Nodes  C  and  3  measure  node  D ’s  forwarding  delay  to  node  7  to  determine  their  pacing  delay  to  node 
7 .  Their  individual  measurements  will  vary  somewhat  but  their  long-term  average  values  are  identical 
because  node  D  uses  the  same  (long-term  average)  pacing  delay  for  transmitting  packets  received  from 
either  node.  Thus  in  steady  state  they  have  the  same  average  pacing  delay  to  node  D .  Likewise,  nodes 
A  and  B  will  have  the  same  average  pacing  delay  to  node  C.  This  implicit  flow  allocation  gives  the 
A  =>  7  and  the  3  =>  7  routes  half  the  bandwidth  of  the  f  =>  7  route.  (All  bandwidth  values  are 
in  packets  per  second  in  units  relative  to  the  bandwidth  on  the  D  =>  7  link.  The  route  bandwidths  are 
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given  in  square  brackets.)  This  network  is  a  generic  example  —  SURAP  only  provides  fair  end-to-end 
flow  allocations  for  networks  with  special  topologies  and  path  losses. 

The  CDMA  end-to-end  flow  control  algorithm,  on  the  other  hand  requires  each  packet  radio  node  to 
compute  a  path  ration,  its  objective  maximum  aggregate  inbound  flow  rate,  by  monitoring  transceiver 
utilization,  CPU  utilization,  buffer  use,  and  output  link  bandwidth.  These  measurements  are  compared  to 
certain  thresholds  to  determine  whether  the  path  ration  should  be  increased  or  decreased.  The  minimum 
path  ration  along  each  route  is  communicated  back  to  its  source  with  a  distributed  algorithm  that  piggy¬ 
backs  rations  in  link-level  control  packets.  (Section  2.2.1)  The  flow  rate  on  each  route  is  then  throttled 
according  to  the  minimum  path  ration.  Note  that  each  node  computes  a  single  path  ration  that  effects 
all  routes  equally.  In  this  way  all  route  sources  sharing  a  bottle  neck  are  throttled  equally  while  those 
route  sources  not  affected  by  the  bottleneck  are  not  throttled.  The  algorithm  recovers  from  node  and  link 
failures  by  throttling  affected  traffic  until  the  problem  is  corrected  or  routes  are  changed. 

A  performance  study[6]  shows  the  algorithm  provides  max/min  fair  flow  control  except  for  certain 
unusual  situations  where  chaimel  contention  creates  dependence  among  neighboring  nodes.  Comparison 
of  the  network  flows  with  and  without  the  algorithm  shows  that  it  maintains  acceptable  delay  without 
significandy  restricting  the  aggregate  network  throughput. 

One  of  the  major  strengths  of  the  algorithm  is  that  it  can  achieve  max/min  fair  end-to-end  flow 
allocation.  This  means  that  only  sources  affected  by  a  bottleneck  are  restricted  and  these  sources  are 
restricted  in  a  fair  marmer.  This  is  particularly  important  for  large  packet-radio  networks  where  unfairness 
could  lead  to  extremely  poor  throughput  for  certain  users  even  though  average  network  throughput  is 
hiffh. 
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Appendix  B.  Adaptive  Link  Gain  Adjustment 


This  algorithm  dynamically  sets  the  bit  rate,  power  level,  and  FEC  rate  on  a  packet-by-packet  basis. 
These  parameter  settings  impact  the  signal-to-noise  ratio  and  signal-processing  gain  at  each  LPR  in  radio 
range;  the  combined  effect  at  a  particular  LPR  is  called  the  link  gain.  The  link  gains  have  a  profound 
impact  on  the  success  of  packet  transmissions  and  thus  network  performance.  Link  gain  is  increased  by 
increasing  the  energy  per  transmitted  bit.  by  using  higher  power,  a  lower  bit  rate,  or  “stronger  coding” 
(lower  FEC  rate).  While  increasing  the  gain  on  any  link  will  always  make  it  more  reliable  and  increase 
its  throughput,  a  unilateral  gain  increase  can  actually  decrease  the  reliability  and  throughput  of  adjacent 
links  by  increasing  mutual  interference.  The  Adaptive  Link  Gain  Adjustment  Algorithm  lets  LPRs 
cooperatively  adjust  their  link  gains  to  make  link  throughput  compromises  that  increase  the  network 
throughput.  SURAP[12]  and  the  CDMA  link-layer(7]  use  slightly  different  versions  of  the  same  basic 
algorithm.  These  differences  do  not  have  a  major  impact  on  our  performance  results. 
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Appendix  C.  LPR  Hardware  Description 

C.l  Channel  Access 

Each  LPR  node  contains  a  half-duplex  radio  transceiver  that  is  always  in  one  of  three  mutually  exclusive 
states:  monitoring,  receiving,  or  transmitting.  Each  packet  transmission  begins  with  a  short  preamble 
that  establishes  bit-synchronization  and  designates  a  pseudo-noise  spreading  sequence  for  receiving  the 
packet.  When  an  LPR  begins  transmitting,  every  LPR  in  radio  range  that  is  monitoring  the  channel 
receives  the  preamble  and  establishes  bit  and  frame  synchronization.  The  preamble  can  designate  one 
of  four  spreading  sequences:  a  common  broadcast  sequence,  either  of  two  “group”  codes,  or  a  special 
receiver-directed  sequence  which  is  different  for  each  LPR  (and  is  a  deterministic  function  of  its  DD). 

When  a  preamble  is  decoded,  the  receiver  attempts  to  de-spread  the  rest  of  the  packet.  If  the  sequence 
it  is  using  doesn’t  match  the  one  used  by  the  transmitter,  the  LPR  loses  synchronization  and  resumes 
charmel  monitoring  or  starts  a  transmission.  Otherwise  it  receives  the  packet  and  stores  it  in  a  buffer.  In 
the  case  of  receiver-directed  transmissions,  each  LPR  receiving  a  packet  will  try  to  use  its  own  unique 
sequence  to  despread  the  signal  —  only  the  intended  receiver  can  succeed,  leaving  the  remaining  LPRs 
free  to  receive  packets  destined  to  them  after  they  quickly  lose  synchronization. 

An  LPR  using  the  CDMA  protocols  has  the  capability  of  either  broadcasting  or  receiver-directing  its 
transmissions  on  a  packet-by-packet  basis.  The  broadcast  mode  is  used  for  distributing  routing  information 
and  for  network  entry.  Data  packets  are  normally  transmitted  with  the  receiver-directed  codes.  But  in 
exceptional  circumstances,  e.g.,  link  failures,  a  packet  can  be  broadcast  so  any  neighbor  closer  to  the 
destination  can  forward  it,  a  procedure  known  as  alternate  routing  that  is  explained  in  Section  (2.2.3). 

C.2  Link  Gain  Settings 

The  LPR  can  control  its  transminer’s  bit  rate  (100  or  400  Kbps),  FEC  rate  (1:1,  7:8,  3:4,  or  1:2),  and 
power  level  (five  watts  with  either  none.  8.  16,  or  24  dB  of  attenuation)  on  a  packet-by-packet  basis. 
These  24  link-gain  senings,  coupled  with  a  128  chip-per-bit  spreading  factor  give  the  LPR  the  capability  of 
counteracting  signal  attenuation,  mutual  interference,  multi-path  reflections,  and  jammers.  The  Adaptive 
Link  Gain  Adjustment  Algorithm  (Section  (B))  implements  a  feedback  loop  that  quickly  changes  these 
settings  in  response  to  environmental  changes. 
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